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ABSTRACT 

We report on the clustering properties of Lyman-break galaxies (LBGs) at z ~ 3. 
The correlation length of flux-limited samples of LBGs depends on their rest-frame 
ultraviolet (UV) luminosity at A ~ 1700 A, with fainter galaxies being less strongly 
clustered in space. We have used three samples with progressively fainter flux limits: 
two extracted from our ground-based survey, and one from the Hubble Deep Fields 
(both North and South). The correlation length decreases by a factor ~ 3 over the 
range of limiting magnitudes that we have probed, namely 25 ^ 1Z ^, 27, suggesting 
that samples with fainter UV luminosity limit include galaxies with smaller mass. We 
have compared the observed scaling properties of the clustering strength with those 
predicted for cold dark matter (CDM) halos and found that: 1) the clustering strength 
of LBGs follows, within the errors, the same scaling law with the volume density as the 
halos; 2) the scaling law predicted for the galaxies using the halos mass spectrum and 
a number of models for the relationship that maps the halos' mass into the galaxies' 
UV luminosity depends only on how tightly mass and UV luminosity correlate, but 
is otherwise insensitive to the details of the models. We interpret these results as 
additional evidence that the strong spatial clustering of LBGs is due to galaxy biasing, 
supporting the theory of biased galaxy formation and gravitational instability as the 
primary physical mechanism for the formation of structure. We have also fitted models 
of the mass-UV luminosity relationship to the data to simultaneously reproduce from 
the CDM halo mass spectrum the dependence of the correlation length with the UV 
luminosity and the luminosity function. We have found that 1) a scale invariant 
relationship between mass and UV luminosity (e.g. a power law) is not supported by 
the observations, suggesting that the properties of star formation of galaxies change 
along the mass spectrum of the observed LBGs; 2) the scatter of the UV luminosity 
of LBGs of given mass must be relatively small for massive LBGs, suggesting that 
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the mass is an important parameter in regulating the activity of star formation in 
these systems, and that the fraction of massive halos at z ~ 3 that are not observed 
in UV-selected surveys is not large. From the fits, for a given choice of the cosmology, 
one can assign a scale of mass to the LBGs. For example, if SI = 0.3 and J7a = 0.7, 
the average mass of galaxies with luminosity 1Z = 23, 25.5 and 27.0 is (M) = 2.5, 0.9, 
and 0.4 x 10 12 M , respectively. The numbers are « 2 times larger and « 10 times 
smaller in an open universe with SI = 0.2 and f^A = and in the Einstein-de Sitter 
cosmologies, respectively. 

Subject headings: cosmology: observations — galaxies: formation — galaxies: evolution 
— galaxies: distances and redshifts 

1. INTRODUCTION 

If cosmic structure has been assembled through the amplification of initial density 
perturbations by gravitational instability, then a general result is that the clustering of virialized 
systems differs from that of the average mass-density distribution, and depends on the properties 
of the systems themselves, such as mass and central density, with more massive systems being 
more strongly clustered in space (e.g. Kaiser 1984; Croft & Efstathiou 1994; Mo & Fukugita 1996; 
Jing & Suto 1996). The hierarchical models of galaxy formation provide a simple description 
of the clustering of virialized structures, or halos, in terms of the "geometrical" bias of their 
spatial distribution relative to that of the average mass-density field (e.g. Kaiser 1984; Peacock & 
Heavens 1985; Bardeen et al. 1986, BBKS), and both analytical works (Matarrese et al. 1997; 
Mann, Peacock & Heavens 1998; Catelan, Matarrese & Porciani 1998; Coles et al. 1998) and 
N-body simulations have consistently confirmed the overall robustness of this treatment (Brainerd 
k Villumsen 1994; Colin, Carlberg, & Couchman 1997; Mo & White 1996; Jing 1999; Katz, 
Hernquist, & Weinberg 1999). 

In this scheme, visible galaxies and clusters have formed when the baryonic gas cooled and 
condensed in the potential wells of the halos (White & Rees 1978), and therefore their spatial 
distribution and clustering properties are predicted to be similar to those of the hosting halos. In 
broad terms, the observations support this interpretation. For example, in both the local universe 
(Davis et al. 1988; Hamilton 1988; Santiago & Da Costa 1990; Park et al. 1994; Loveday et al. 
1995; Tucker et al. 1997; Valotto & Lambdas 1997) and at intermediate redshifts, e.g. z ^ 1 (Le 
Fevre et al. 1996; Carlberg et al. 1997; Cohen et al. 2000), the clustering strength of galaxies 
has been observed to depend on properties of the galaxies that correlate with the mass, such 
as the optical luminosity and the spectral types, with more luminous and earlier types having 
stronger spatial clustering than fainter and later ones. For clusters of galaxies a similar trend 
with the richness parameter has also been observed (e.g. Bahcall & West 1992; Bahcall & Chen 
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1994). These results have generally been interpreted as evidence of the existence of galaxy biasing 
(Peacock 1997). 

A number of works have attempted to test this paradigm by comparing its predictions to 
the properties of galaxies, using semi-analytical models or N-body simulations (for a review see, 
among others, Frenk & White 1991; Kauffmann, Nusser & Steinmetz 1997; Baugh et al. 1998; 
Governato et al. 1998; Katz, Hernquist & Weinberg 1999; Somerville, Primack & Faber 2000; 
Mo, Mao & White 1999). An essential assumption of these works is the relationship between the 
activity of star formation of nascent galaxies and the properties of the hosting dark matter halos, 
such as the mass, central density and angular momentum (e.g. Dalcanton, Spergel & Summers 
1997; Heavens Sz Jimenez 1999). This relationship is key to the predicted properties of the galaxies 
in the models, but unfortunately, in contrast to the relative conceptual simplicity required to 
model the dark matter, the relevant physics is very complex and little empirical information is 
available to help constraining it. Until recently, the models have been tested against observations 
in the local and intermediate redshift universe. This, however, requires modeling the evolution of 
the galaxies from the epoch of formation to that of the observations. The properties of relatively 
evolved galaxies are, in general, the result of rather complicated processes of transformation (e.g. 
merging, interactions and environmental effects), and they can have decoupled from those of the 
systems in which they (or their components) originally formed. 

A more straightforward test can now be attempted directly at high redshifts, thanks to the 
large and homogeneous samples of star-forming galaxies at z > 2 made available by the highly 
efficient Lyman-break technique (Steidel et al. 1996; Madau et al. 1996; Steidel et al. 1999, S99). 
These "Lyman-break galaxies" (LBGs) are selected from their UV emission properties, which 
primarily reflects the instantaneous rate of star formation (modulo obscuration by dust). This 
offers the distinct advantage of working with a well defined physical quantity, which is relatively 
independent of the previous evolutionary history of the galaxies. 

The samples are large enough that quantitative studies of clustering, e.g. low-order statistics 
such as the correlation function, have become possible. One of the most interesting results from 
these works is that LBGs are strongly clustered in space, with a correlation length comparable 
to that of present-day galaxies and larger than that of galaxies at intermediate redshifts (Steidel 
et al. 1998; Giavalisco et al. 1998; Adelberger et al. 1998, in the following S98, G98 and A98, 
respectively. See also Connolly, Szalay & Brunner 1998, and Arnouts et al. 1999). This is difficult 
to explain in terms of the clustering of the mass for any reasonable assumptions of the parameters 
of the background cosmology for typical CDM power spectra, if the normalization of spectrum has 
to match the present-day distribution of galaxies and clusters (e.g. Eke et al. 1995). 

The strong spatial clustering of the LBGs and the apparently non-monotonic evolution with 
redshift of the galaxy correlation length (G98; Connolly et al. 1998) seem to be generally consistent 
with the expectations of the theory of biased galaxy formation (S98; G98; A98; Connolly et al. 
1998; Peacock et al. 1999), apparently implying that they form in relatively massive halos. If 
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halos and LBGs are physically associated, an obvious consequence is that the relationship between 
the clustering properties and the spatial abundance of the galaxies is determined by that of the 
halos, a fact that in principle can be tested. In this paper we present a measure of the scaling law 
of the clustering strength with the UV luminosity and with the volume density of LBGs at z ~ 3 
and discuss the comparison with the predictions of the gravitational instability theory for CDM 
halos. Specifically, we present the measure the correlation length and the volume density of three 
samples of LBGs with different flux limits; the two brighter samples come from our ground-based 
survey, while the faintest one comes from the Hubble Deep Fields (both North and South, HDF 
hereafter). 

The problem in comparing the observations with the theory is that one generally does not 
know how the properties of the halos (e.g. the mass) map into those of the observed galaxies 
(e.g. the UV luminosity). Therefore, with the help of a very simple but relatively general model 
we explore if, by requiring that the theory simultaneously reproduces the observed scaling law of 
the clustering strength with the UV luminosity and the luminosity function (our observables) , 
one can derive some general conclusions about the nature of the clustering properties of LBGs as 
well as information about the association between the UV luminosity and the mass. Interestingly, 
the scaling law of the clustering strength with the volume density is largely insensitive to the 
assumptions of the model — it only depends on how tightly the UV luminosity correlates with the 
mass of the galaxies — suggesting that one can use the clustering of LBGs to test in broad lines 
our ideas on structure formation and the physics of star formation in high redshift galaxies. 

We consider three cosmological models in the paper, an open universe with matter density 
parameter 12 = 0.2 and cosmological constant density parameter S7a = 0, a spatially flat, 
low-density universe with li = 0.3 and 12a = 0.7, and the Einstein-De Sitter universe with = 1 
and J7a = 0. Whenever we compute a physical quantity that depends on the cosmology, we list the 
three numbers that correspond to these three cosmologies in the same order as we have presented 
them here. Throughout the paper we use H = 100 h km s" 1 Mpc" 1 . 

2. THE THREE SAMPLES OF LYMAN BREAK GALAXIES 

The high efficiency of the Lyman-break technique at z ~ 3, and the relative ease with 
which its selection criteria can be quantified and modeled, make it particularly advantageous for 
constructing large and well controlled samples of high-redshift galaxies that are nicely suited to 
study galaxy clustering (see the discussion in G98 and S99). The technique and its application to 
clustering studies have been extensively discussed elsewhere (Steidel, Pettini & Hamilton 1995; 
Madau et al. 1996; Dickinson 1998; S98; G98; A98), and here we only review the aspects that are 
relevant to this paper. 

We shall now discuss the properties of the three samples of LBGs at z ~ 3, or "[/-band 
dropouts", that have been used in this study, which span a range of limiting magnitudes 
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Ami « 2.2 mag. The two brighter samples are selected from our U n GTZ ground-based survey 
(S99) and considerably overlap with those discussed by G98 and A98. The fainter one comes 
from the HDF surveys, both the North and South (Williams et al. 1996; Williams et al. 2000; 
Casertano et al. 2000). The brighter of the two ground-based samples, referred to here as 
the "spectroscopic sample" (SPEC), consists only of galaxies with secured redshifts, while the 
fainter one, or "photometric sample" (PHOT), includes all the [/-band dropouts (with or without 
redshifts) down to limiting magnitudes ft < 25.5 (all magnitudes in this paper are in the AB 
scale of Oke & Gunn, 1983). The HDF sample is built in an analogous way and includes all the 
[/300-band dropouts down to V&q§ < 27. At the time of this writing, there are 24 spectroscopic 
redshift identifications in the HDF-N sample (see Dickinson 1998), and 3 in the HDF-S (Cristiani 
et al. 2000). Table 1 lists the relevant parameters of the three samples. 



2.1. The Ground Based Samples 

As in previous studies of the clustering properties of LBGs (S98, G98 and A98), here too an 
object is considered a Lyman-break galaxy if its colors satisfy the criteria 

(U n - G) > 1.0 + (G-ft); (E/„- G) > 1.6; (G - ft) < 1.2, (2.1) 

with the additional requirement ft < 25.5 imposed to produce a reasonably complete sample. As 
discussed by S99, the definition of color criteria to select LBGs is, to some extent, arbitrary. The 
definition above is a relatively stringent color cut, and other criteria can certainly be defined that 
would result in larger samples of high-redshift galaxies. However, these would also contain a non 
negligible number of interlopers at lower redshifts. Since for two of the three samples discussed 
here we measure the correlation length by inverting the angular correlation function, interlopers 
provide a source of systematic errors that would bias our measures, and need to be minimized. 
With the spectroscopic information available, we have defined Eqn.2.1 in order to obtain an 
optimal balance between the competing requirements of having as large a sample as possible which 
is also as free of low-redshift interlopers as possible, i.e. maximizing the efficiency. In the above 
case, the only significant source of interlopers are galactic stars (« 3.4%), and all the galaxies that 
satisfy Eqn.2.1 and that have been confirmed spectroscopically were found to have redshift in the 
expected range for C7 n -band dropouts, i.e. 2.2 z ^ 3.8. LBG candidates that remain unidentified 
have spectra with S/N too low to allow a secure measure of the redshifts, and in no case was an 
identified redshift found outside the range expected for LBGs. 

At the time of this writing our ground-based survey includes 1,243 candidates from 15 
different fields that satisfy Eqn.2.1. We refer to this as the TOTAL sample. We have secured 
redshifts for 583 sources from the TOTAL sample. Of these, 20 are stars (3.4%), while the 
remaining 563 are high-redshift sources. These high-redshift objects form the TOTALSPEC 
sample, which includes 547 galaxies, 8 QSOs and 8 other types of AGNs. Figure 1 plots the 
redshift histogram of the 547 galaxies of the TOTALSPEC sample, after exclusion of the QSOs 
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and AGNs. The two ground-based samples considered in this paper are defined as follows. 

The PHOT sample includes the 876 LBGs (either candidates or galaxies with secured redshifts) 
of the TOTAL sample extracted from the 7 largest and deepest fields of the survey as detailed 
by G98, who used the sample to measure the angular correlation function of LBGs. The average 
surface density of LBGs with 1Z < 25.5 from these 7 fields and its standard deviation are 

S P = 1.22 ± 0.18 arcmin^ 2 (2.2) 

The SPEC sample consists of the 268 galaxies with secured redshift identification discussed 
in A98 plus additional 178 new redshifts from the field 1415+527 (the "Westphal field" in the 
nomenclature of G98). As before, we have selected these 446 galaxies because they come from our 
largest and deepest fields, which also have the highest degree of spectroscopic completeness. Thus, 
the SPEC sample is included in the PHOT sample, and consists of all its galaxies with measured 
redshifts. A98 used the initial sample of 268 galaxies to estimate the correlation length with the 
counts-in-cell analysis. With the new 178 redshifts from the Westphal field, we have obtained 
an additional data point for such statistics, which we have averaged, after proper weighting, to 
the previous measure as we will discuss later. The redshift histogram of the Westphal sample is 
plotted in Figure 2 together with the smoothed histogram of the TOTALSPEC sample. 

We shall now compare the relative depth of the PHOT and SPEC samples, and assign an 
equivalent depth to the SPEC sample, in terms of the magnitude of the galaxies that numerically 
dominate it, in order to estimate its surface density. In the course of the spectroscopic follow-up 
of the Lyman-break survey, the selection of the targets from the pool of available candidates was 
primarily designed to optimize the size of the spectroscopic sample. Essentially, this meant cutting 
as many slits as possible in any one mask of the LRIS multi-object spectrograph (Oke et al. 1995) 
to correspond to the brightest possible candidates, in order to increase the chances of positive 
identification. As a result, the SPEC sample has been subject to an additional flux selection that 
has made it overall brighter than the PHOT sample. Although strictly speaking the SPEC sample 
is not flux-limited (galaxies as faint as 1Z ~ 25.5 can still be counted in it), this preferential 
selection of brighter galaxies makes the magnitude distribution equivalent, for practical purposes, 
to that of a traditional flux limited sample, as it can be seen in Figure 1. 

We have estimated the depth and surface density of the SPEC sample by comparing its 
magnitude distribution N(m) to that of the PHOT sample. First, we assigned a depth to both 
samples using the magnitude at which the number counts reach a maximum. We have identified 
the location of the maximum by arranging the magnitudes in a vector sorted by ascending 
order and running a boxcar window on this vector to derive the surface density of galaxies as a 
continuous, smooth function of the magnitude 1Z. This method has the advantage of not requiring 
any binning of the data and is relatively insensitive to the size of the boxcar window. For the 
SPEC sample, we find the maximum to be located at TZ e ff = 24.7 ± 0.1, where the error bar 
reflects the uncertainties induced by varying the size of the boxcar window and the photometric 
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error. The maximum of the PHOT sample is located at TZ e ff = 25.1 ± 0.1, namely 0.4 mag 
fainter. Subsequently, we have assigned the SPEC sample the surface density of a flux-limited 
sample of LBGs with R e ff = 24.6. We have estimated this value assuming that the magnitude 
distributions of the SPEC and PHOT samples have the same shape, which is approximately the 
case as shown by Figure 3. In this case we can scale the size of the PHOT sample by the ratio 
of the integral counts Nxim) = f™^ N(m') dm' evaluated at two magnitudes that differ from 
each other by Am = 0.4, namely the difference between the values of m e ff of the two samples. 
We have computed this ratio using a linear fit of the Log(N) vs. m relationship in the range 
22.5 < 1Z < 24.7, which is a magnitude range relatively unaffected by flux incompleteness. Using 
this method, we have found the surface density of the SPEC sample to be 

£ s = 0.7±0.1 arcmin- 2 , (2.3) 

or 0.6 x smaller than the surface density of the PHOT sample. For practical purposes, the SPEC 
sample can be considered equivalent to a flux-limited sample with limiting magnitude 1Z £j 25.1. 
Figure 3 shows the magnitude distribution of the three samples. 

2.2. The HDF Sample 

The faintest sample that we have considered comes from the HDF survey and includes 
Lyman-break galaxies from both the Northern (HDF-N) and Southern (HDF-S) WFPC2 fields. 
We have used the STScI versions v2.0 and vl.O of the stacked and drizzled images in the E/300, 
-B450, V(306 and hu filters of the HDF-N and HDF-S, respectively, to build the sample, retaining 
only the data from the WF chips, and neglecting the small area covered by the PC chip. 

We have constructed samples of LBGs with criteria essentially identical to those adopted for 
the ground-based survey. The photometric systems used in the HDF survey, however, is somewhat 
different from the one adopted for the ground-based survey and, as a result, the resulting redshift 
distribution of the HDF LBGs differs from that of their ground-based counterparts. The most 
conspicuous difference is that the C/300 bandpass is significantly bluer than the U n one, causing 
Lyman-break galaxies start to enter the selection window built with this bandpass already at 
redshifts z £ 1.8. By contrast, the ground-based bandpass U n starts to effectively pick up LBGs 
when they have redshifts in excess of z £ 2.2. 

As we pointed out earlier, there is some degree of arbitrariness in setting the color cuts that 
define LBGs, and a number of samples of Lyman-break galaxies in the HDF (so far, essentially 
in the North field only) have been proposed by several groups. The color selection criteria differ, 
and no "definitive" method has yet been established. The number of LBG candidates therefore 
varies from sample to sample depending on the adopted criteria. For example, Madau et al. 
(1996) defined conservative criteria based on models of galaxy color distributions in order to select 
z > 2 galaxies while avoiding significant risk of contamination from objects at lower redshifts. 
These criteria, however, miss some of the galaxies that have been spectroscopically confirmed to 
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have z £ 2. Using the wealth of redshift information available in the HDF (Steidel et al. 1996b; 
Cohen et al. 1996; Lowenthal et al. 1997; Dickinson 1998) to fine-tune the selection, Dickinson 
(1998) built a sample of LBGs from the HDF-N by applying the so called "marginal" criteria 
proposed by Steidel, Pettini h, Hamilton (1995), which for the HDF filters are re- written as 
(^300 — -B450) > 1-0 + (-B450 — ^606) and (-B450 — V606) < 1-2. This selection window successfully 
recovers all the 27 spectroscopically confirmed LBGs in the HDF at 2 < z < 3.5. As is the case for 
the ground-based sample, the dominant source of contaminants (always very modest in absolute 
terms) are galactic stars, but these are easily recognized in the WFPC2 images and excluded 
(all obvious stars with the above colors have also already been observed spectroscopically, as it 
turns out). Excluding the stars, there are 198 galaxies in the HDF-N and 222 in the HDF-S with 
V(306 < 27.0 that satisfy the above selection criteria. 

The above selection window, however, is slightly different from that adopted for the 
ground-based samples, because it lacks the additional cut ([/300 — -B450) > 1-6. While such a 
difference is relatively minor, we have decided to include it in our definition of LBGs in the HDF 
for consistency with the ground-based sample. Therefore, the color selection criteria that we have 
adopted for the HDF sample are: 

(^300 - £450) > 1.0 + (£ 450 " ^606); (^300 - £450) > 1.6; (B 450 ~ Veoe) < 1.2, (2.4), 

with the additional limit Vej06 < 27.0 imposed to produce a sample reasonably complete in 
apparent magnitude. This selection window recovers 23 of the 24 spectroscopic redshifts of the 
HDF-N and the 3 ones of the HDF-S, and yields a sample of 126 galaxies in the HDF-N and 145 
in the HDF-S, or a total sample of 271 galaxies. There are actually a few more LBG candidates 
in both HDF fields that satisfy Eqn.2.4. However, in the present study, we have considered only 
LBGs from those regions of the HDF WF chips that received the maximum exposure time, i.e. we 
have excluded the edges around each chip where the S/N was lower due to the dithering and the 
incompleteness is difficult to correct. The surveyed area is 4.45 arcmin 2 in the North and 4.76 
arcmin 2 in the South, or a total area of 9.21 arcmin 2 . Taking the average of the surface densities 
of the North and South field we find the surface density of LBGs to be 

£ H = 29.4 ±3.5 arcmin^ 2 , (2.5) 

where the error bar is quadratic sum of the two individual Poisson error bars. Finally, we can 
also put the HDF galaxies on the same magnitude scale of the ground-based ones using the 
approximation 1Z = (Vqqq + Isu)/^), proposed by Steidel et al. (1996b; see also S99). The 
distribution of the 1Z magnitudes derived in this way is shown in Figure 3 with dotted data points. 

3. CORRECTING THE VOLUME DENSITY FOR INCOMPLETENESS 

In general, samples of LBGs will not include all the star-forming galaxies physically present 
in the volumes of space probed by the survey, even if their luminosity is high enough for 
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detection. Galaxies brighter than the magnitude limit will be missing from the samples because 
of photometric errors, which scatter their colors outside of the selection window, confusion 
blending with nearby sources, and because they have intrinsic colors that do not satisfy the 
color selection criteria. For the ground-based samples, we have quantified this incompleteness 
with Monte-Carlo simulations in which large numbers of artificial LBGs with a range of intrinsic 
colors and magnitudes were added to the original CCD frames and then recovered using the same 
techniques adopted for the real galaxies. A complete discussion of the results of these simulations, 
including estimates of the intrinsic distribution functions, will be presented in a forthcoming paper 
(Adelberger et al. in preparation). Here we use the initial results of this work, which in part have 
already been discussed in S99, that have allowed us to correct the LBG samples for incompleteness 
with the goal to estimate their volume density. 

The main output from the simulations is the probability p(m, z) that a LBG at redshift z 
with apparent magnitude m and colors extracted from the intrinsic distribution of galaxy spectral 
shapes observed in LBGs at z ~ 3 is recorded in our sample (see S99 for the tabulated values of 
the function p(m,z) and relevant discussion). For the purposes of this paper it is convenient to 
integrate p(m, z) over the magnitudes and quantify the completeness of a sample with m < mn m 
by means of a parameter as: 

JV(z) = a x N{z), (3.1) 

where N{z) and N{z) are the predicted and observed redshift distributions, respectively. With 
good approximation a does not depend on the redshifts, and from the simulations we found 
a ~ 0.55 a,t1Z = 25.5 (a is larger at brighter magnitudes). The effective cosmic volume comprised 
by the LBG samples is easily computed as the integral of the comoving volume element dV(z) 
occupied by the galaxies with redshift in the range [z,z + dz] per arcmin 2 , weighting it by the 
probability that these galaxies are included in the sample, namely: 



V eff = f(z)dV(z), (3.2) 
Jo 

where f(z) = a N(z)/Np ea f ( .. Finally, if £ is the surface density of LBGs, the volume density is 
simply given by 

n=-^-. (3.3) 

V eff 



3.1. The Ground Based Samples 

The redshift distribution N{z) of the 547 galaxies in the TOTALSPEC sample (from which 
we have excluded the 18 QSOs and AGNs) is shown in Figure 1, and a smoothed curve is shown in 
Figure 2. The mean value is z = 3.04 and the standard deviation is a z = 0.24, with approximately 
90% of the galaxies included in the range 2.57 ^ z ^ 3.37, and none with z < 2.3. This is rather 
similar to the one shown in G98, but it benefits from ~ 45% more redshifts. When we use this 
redshift distribution in the Limber transform of Eqn.4.5 (see later), we perform a local cubic spline 
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interpolation on the histogram. In each case, the final results do not appreciably depend on the 
choice of the binning used to build the histogram. 

We assume that the function N(z) of Figures 1 and 2 is representative of both the SPEC and 
PHOT samples. In the case of the SPEC sample such an assumption is actually unnecessary, since 
this sample is contained in the TOTALSPEC one, and the galaxies in TOTALSPEC that are not 
included in SPEC come from fields that were not used in the counts-in-cell statistics only because 
they either were too small or had poor spectroscopic coverage. 

The case of PHOT is different. This sample has only ~ 50% spectroscopic completeness and, 
as discussed above, is fainter, with ~ 50% of the still spectroscopically unidentified galaxies having 
TZ > 25 mag. Therefore, there is the possibility that its redshift distribution might differ from 
that of the SPEC sample. However, we believe that this is quite unlikely. As we have already 
pointed out in G98, our ability of securing redshifts for the LBGs of Eqn.2.1 has no obvious 
dependence on the magnitude (up to 1Z ~ 25.5) or on the color of the galaxies. To test this, we 
have divided the TOTALSPEC sample into 2 sub-samples, a bright one which includes all galaxies 
with 1Z < 24.5, and a faint one, which contains the remaining galaxies. The redshift histograms 
of these 2 sub-samples are plotted in Figure 1 with thin dotted and dashed lines, respectively. 
As it can be seen from the figure, the 2 histograms are essentially identical, with mean redshifts 
zb = 3.021 and zp = 3.016, and standard deviations ap = 0.233 and op = 0.242, respectively. A 
Kolomgorov-Smirnov test shows that the probability that the 2 distributions are not extracted 
from the same parent distribution is only ~ 42%. In practice, because both the volume density 
and (as we shall see) the Limber transform are rather insensitive to small changes in the shape of 
N(z), we have assumed that the PHOT and SPEC samples have the same redshift distribution 
and that this is described by the histogram shown in Figure 1 (or the smooth curve in Figure 2). 

Using Eqn.3.3 we measured the volume density of the PHOT sample to be 

n P = (1.9 ±0.3, 1.8 ±0.3, 7.2 ± 1.1) x a" 1 x 10~ 3 /j- 3 Mpc 3 (3.4) 

for the three cosmological models considered in this paper, respectively (see Section 1). The 
volume density of the SPEC sample is 1.8 x smaller and equal to 

n s = (1.1 ±0.2, 1.0 ±0.2, 4.0 ±0.6) x a" 1 x 10~ 3 h~ 3 Mpc 3 , (3.5) 

Table 2 lists the volume densities of the two ground-based sample for the values of a used here, 
namely a = 0.55. 

3.2. The HDF Samples 

With only 27 redshifts secured so far, the redshift distribution of the HDF samples is much 
less accurately constrained than that of the ground-based survey, which results in relatively larger 
uncertainties on its volume density and correlation length. The histogram of the HDF-N redshifts 
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is shown in Figure 1 as a shaded area together with that of the ground-based sample. There 
are at most 5 galaxies in any of the redshift bins and only one galaxy each in the z = 2.6 and 
z = 2.8 bins, and the shape of N{z) is unlikely to reflect the redshift distribution for the adopted 
selection criteria. The observed N{z) most likely results from 1) small number statistics; 2) 
incompleteness at z ^ 2.5 due to the difficulty of measuring the relevant spectral features with the 
LRIS spectrograph; and, more importantly, 3) to fluctuations introduced by the spatial clustering. 
Moreover, the secured redshifts are of galaxies belonging to the bright end of the luminosity 
distribution, whereas the N(z) and the volume density of the sample are actually dominated 
by the galaxies close to the magnitude limit. Spectroscopic redshifts for these galaxies are not 
practical with current instrumentation, and although at the depth of the ground-based sample 
there is no evidence of a dependence of N(z) with the luminosity, we also do not know if such an 
effect is present at fainter fluxes. 

Despite this paucity of spectroscopy for the HDF LBGs, we can still obtain some constrain 
on their N(z) by using the information on the intrinsic color distribution of the ground-based 
LBGs, as discussed in S99. This will allow us to derive the volume density and the correlation 
length of the HDF sample with enough precision for the purposes of this paper. Specifically, we 
have used Monte-Carlo simulations to compute the expected redshift distribution of the C/300 
dropouts combining the ground-based intrinsic color distribution LBGs (under the assumption 
that it does not depend on the luminosity) with the color criteria of Eqn.2.4. This technique, 
discussed in detail in S99, also assumes that the volume density of LBG does not appreciably 
change over the probed redshift range. This is very likely a minor assumption, as suggested by the 
fact that the same technique is very successful in reproducing the observed redshift distributions 
of both the U n - and G-band dropouts from the ground-based survey (see. Figure 4 of S99). The 
predicted N(z) for the HDF is shown in Figure 7 of S99 and is also reproduced here in Figure 1 as 
a thin continuous line with arbitrary normalization. As expected, it extends to smaller redshifts 
compared to the ground-based one, although the two distributions are very similar at the high 
redshift end. The HDF one has mean redshift zh = 2.60 and standard deviation an = 0.69. 

To estimate the degree to which the volume densities and (as we shall see later) the spatial 
correlation length depend on the shape of N(z), we have also considered two additional redshift 
distribution functions for the HDF sample. We considered 1) a top-hat function over the range 
2.0 < z < 3.5, with mean redshift and standard deviation equal to zth = 2.75 and <jth = 0.43. 
This is the redshift distribution originally assumed for the LBGs in the HDF-North by Madau 
et al. (1996); 2) a synthetic function built by summing together 5 modified versions of the 
ground-based N(z), each obtained by rigidly shifting it by the amount Az = 0.0, —0.2, —0.4, 
—0.6, —0.8, respectively. This last function has mean redshift zsyn = 2.61 and standard deviation 
&SYN = 0.38, and while it is qualitatively similar to both the fiducial and ground-based N(z)'s, 
it is narrower than the former and broader than the latter. Its mean redshift is the same as the 
fiducial function, consistent with the expectations given the HDF filters. Table 2 lists the volume 
densities computed from Eqn.3.3 with a = 1 for the three adopted cosmologies. It can be seen 
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from the table that changing the N(z) among those considered in our pool results in £ 30% 
variations of the volume density. 

Note that, while we have included the effects of the intrinsic variations of the colors of LBGs 
when we have derived the expected redshift distribution of the HDF sample (and hence these 
effects have been accounted for in the measure of the volume density), we have not performed the 
Monte-Carlo simulations on the HDF images to determine the extent of the incompleteness due 
to photometric errors and crowding effects. This is very likely no cause of systematic errors in the 
measure of the angular correlation function, but it affects the measure of the volume density by 
causing it to be biased low. However, we expect this incompleteness to be significantly less severe 
than in the ground-based samples, as empirically supported by the good agreement between the 
luminosity function of the ground-based sample of LBGs (which contains the incompleteness 
corrections due to photometric errors and crowding effects) and that of the HDF North sample 
discussed by S99 (see their Figure 7a). The reason is that while photometric accuracy at the faint 
end of the HDF sample is comparable to that of the ground based one, the WFPC2 images have 
significantly higher angular resolution which greatly reduces the incompleteness due to crowding 
effects. We will quantify the incompleteness of the HDF samples in future works. For the moment 
we note that, strictly speaking, the measures of the volume density of the HDF sample must be 
considered a lower limit. As will become clear later, however, this will not significantly affect our 
conclusions. 



4. THE CLUSTERING OF LYMAN BREAK GALAXIES 

In previous works we have measured the spatial correlation function of Lyman-break galaxies 
using the counts-in-cell technique for the SPEC sample ( A98) , and the inversion of the angular 
correlation function w(0) for the PHOT sample (G98). In this section we present the measure 
of the angular correlation function of the HDF sample and its inversion to estimate the spatial 
correlation length. We also present minor revisions to the previous measures using additional data. 



4.1. The SPEC Sample 

The variance a 2 g of galaxy counts in cells of assigned volume is proportional to the integral of 
the spatial correlation function £ s (r) over the volume of the cell 

°l = VT I I dV 1 dV 2 i g {r 12 ) (4.1). 

V cell J J Vcell 

Assuming the power-law model £ s (r) = (r/r )~' y , one can estimate the correlation length r from 



Since we have augmented the sample discussed in A98 with an additional 178 new redshifts 



oi if one knows (or assumes) the value of the slope 7. 
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from the Westphal field 1415+527 (the 446 total galaxies now comprise the SPEC sample), we have 
revised the measure of ro. Figure 2 shows the distribution of the new redshifts in the Westphal 
field together with the redshift selection function of the survey. The measure by A98 consists of 
a weighted average of a g estimated from cubic redshift cells of size 11.9, 11.4, 7.7 /i _1 Mpc in 
6 individual fields that have the same geometry (~ 9 arcmin in side). Since the Westphal field 
is considerably larger, we could not simply measure the corresponding a g and include it in the 
average. Therefore, we have first derived ro and then averaged this value with the other analogous 
measures. We have analyzed the Westphal field with the same technique described in A98. We 
have divided the sample into a grid of roughly cubic cells whose transverse size is equal to the 
field of view (now ~ 15.1 arcmin), which at z = 3 corresponds to ~ 20.4, 19.7 and 13.2 ft. Mpc, 
and estimated the count variance using both the two estimators discussed by A98 with consistent 
results^. We found = O.ljftof, 0.22±g;? 8 , 0.12±g;g| (1-a error bars). 

Using the power-law model and by approximating the cubes with spheres of equal volume (this 
considerably simplifies the algebra, but has little effect on the final results), the count fluctuations 
can be expressed in terms of the correlation length ro as a g = 72 (ro/i? ce zz) 7 /[(3 — 7) (4 — 7) (6— 7) 2 7 ] 
(Peebles 1980, §59), with R ce u the radius of the equivalent sphere. Assuming 7 = 2.0 ± 0.2 
(the value of the slope measured from the PHOT sample, as we shall see later), we found the 
correlation length of the Westphal sample to be ro = 2.7 ± 0.7, 3.8 ± 1 and 2.1 ± 0.6 /i _1 Mpc. 
After inverse-variance weighting with the analogous values by A98, ro = 5.6 db 0.8, 5.4 ± 0.8 and 
3.6 ± 0.6 h^Mpc using our value of 7, we have finally derived the correlation length of the SPEC 
sample: 

r = 4.2±0.6, 5.0 ±0.7, 3.1 ± 0.4 ft^Mpc. (4.2) 

A smaller slope results in a larger correlation length, and the uncertainty due to the error on 7 is 
comparable to that due to the error on a g (we added the two contributions in quadrature). Thus, 
the Westphal field is "less correlated" (albeit at the ~ la level) than the average (from the other 
6 fields), very likely a manifestation of the still poorly quantified cosmic variance, although the 
values in Eqn.4.2 are within the errors of the measures by A98. Finally, it is important to keep in 
mind that with this technique one measures the average of the correlation function over a volume 
that is ~ 5 times larger than the correlation volume. The method, therefore, cannot provide any 
information on the shape of £(r). Only with the assumption of the power law model and the value 
of its slope one can actually derive a value of the correlation length. 



2 A98 used two estimators of a 2 ., a non-parametric one denned as S = [(N — p) 2 — p]/n 2 , where fi is the mean 
number of galaxies in the cell and N is the number actually found, and a maximum-likelihood estimator where the 
distribution function of the galaxy fluctuation field 8 g = (p g — p g )/p g is assumed to be lognormal (cf. their Eqn.7 
and Figure 3). 
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4.2. The PHOT and HDF Samples 

In G98 we presented the measure of tq from the PHOT sample, obtained by inverting the 
angular correlation function w(9) with the Limber transformation and the observed redshift 
distribution function. The same technique is employed here for the HDF sample. We used the 
estimator of w(9) proposed by Landy & Szalay (1993) 

... DD(9)-2DR{9) + RR{9) 

w{9) = rW) 

which minimizes the variance of the estimate (see also the discussion by Hamilton 1993). 
Here DD(9) is the number of pairs of observed galaxies with angular separations in the range 
(9,9 + 69), RR(9) is the analogous quantity for homogeneous (random) catalogs with the same 
geometry of the observed catalog, and DR(9) is the number of observed-random cross pairs. This 
statistics produces an estimate of w(9) which is biased low by a factor (the "integral constraint") 
/ ~ 1 + O({9 /9 max )P), where w(0 o ) = 1 and (3 ~ 1 (Peebles 1974), but in the case of the PHOT 
sample the correlation amplitude is rather weak and 9q <C # m ax, and hence we can neglect this 
small correction. This bias can be significant in the HDF, however, because of its small areal 
coverage, and we have estimated its magnitude using numerical simulations, as we describe later. 

For both the PHOT and HDF samples we measured w(9) from a weighted average of the 
individual w(9) functions of the fields that comprise each sample using inverse variance weighting; 
it made little difference if we used Poisson or bootstrap variance (Ling, Barrow &i Frenk 1986; 
see G98 for a a complete discussion). In the HDF we have estimated w(9) separately from the 
whole mosaic of WF CCDs in each of the Northern and Southern fields and then averaged the two 
measures together. We have chosen not to split each HDF sub-sample according to the galaxies' 
positions in the 3 CCDs that cover the WFPC2 field of view and then average the w(9) functions 
from each CCD. The advantage of this choice is that one maximizes the S/N due to the larger 
number of galaxy-galaxy pairs present in this case, an important consideration with such a small 
sample as the HDF (one looses all the inter-CCD pairs by considering each CCD separately). The 
disadvantage is that if there are slight variations of surface density in the LBG samples from the 
various CCDs (as the result of slight variation of sensitivity from chip to chip), then these will 
produce a spurious clustering signal. At the limiting magnitude of our sample there is no evidence 
for such variations, as we have verified by checking the surface density of LBG candidates and 
the S/N of the photometry in each chip from both the HDF-N and HDF-S. The fluctuations are 
consistent with the Poisson statistics, with no obvious indications that one CCD is preferentially 
detecting more candidates then the other ones. Therefore, we opted to work with the whole 
mosaic, after masking the regions that received less than the maximum amount of exposure time 
due to dithering. 

We subsequently fitted the weighted average to the power law w(9) = A UJ 9~^ with Levenberg- 
Marquardt non- linear least-squares (Press et al. 1992). To estimated confidence intervals on the 
parameters A u and [3, we generated a large ensemble of Monte-Carlo realizations (100,000) of the 



(4.3) 
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measured w(6), assuming normal errors, and calculated best fit parameter values for each of these 
synthetic data sets (e.g. Press et al. 1992 §15.6). As discussed in G98, we found that the fitted 
parameters slightly depend on the choice of the binning used to compute w(9), and to take this 
additional source of uncertainties into account we have included the effects of a randomly variable 
binning into the Monte-Carlo simulations. The error bars, therefore, reflect the uncertainty that 
derives from the choice of the binning. Figure 4 shows the measured w{9) of the PHOT and HDF 
samples plotted together with the best fit power-law model. For clarity the error bars of w(6) have 
been plotted separately from the data points. 

Finally, we derived the spatial correlation function £(r) by inverting w(9) with the Limber 
transformation, using the fiducial HDF redshift distribution N(z) (Peebles 1980; Efstathiou et 
al. 1991). If the spatial function is modeled as 

£(r) = (r/r )- 7 x F(z), (4.4) 

where F(z) describes its redshift dependence, the angular function has the form w{6) = A w 6~@, 
where = 7 — 1 and 

A w = Crl [ Zf F{z) D]" 1 (z) N{zf g{z) dz x H N(z) dz 

J Zi J Zi 

(Efstathiou et al. 1991). Here Dq(z) is the angular diameter distance, 



(4.5) 



g(z) = ^[(1 + z)\l + n z + fi A ((l + zy 2 - (4.6) 

c 

and C is a numerical factor given by 

r[( 7 -i)/2] 

c = ^ r( 7 /2) • (47) 

The Lyman-break galaxies' redshift distribution is considerably narrower than those of traditional 
flux-limited redshift surveys and it is reasonable to expect little evolution of their clustering over 
so short a range. In this case the function F{z) can be taken out of the integral and the quantity 
tq(z) = tqF(z) is the correlation length at the epoch of observations. 

The slope of the HDF sample is 

"/hdf = 2.2t 3 2 (4.8), 

within the errors the same value found in the ground-based sample. The correlation length of the 
HDF sample obtained with the fiducial N(z) in the Limber transform is 

r ,HDF = 1.118:1, 1.2181, 0.8l™ ft^Mpc, (4.9) 

respectively, approximately 3 times smaller than the values found in the PHOT sample. We have 
estimate the uncertainty on the measure of tq from the inversion of the Monte-Carlo realizations 
oiw{6). 
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With the total number of redshift in the SPEC sample increased to 547 from the original 376 
discussed in G98, we have also recomputed the Limber inversion of the PHOT sample using the 
updated N(z). Using the same procedure described above for the HDF, we have found 

7PHO T = 2.0+° 2 2 (4.10) 

and 

r ,PHOT = 3.lt°;£, 3.2lg;?, 1.9lg;t h^Mpc (4.11) 

in the three cosmologies, respectively, in very good agreement with the values presented by G98. 
Table 3 lists the measures of the slope of the correlation function 7 and of the spatial correlation 
length ro for the three samples in each of the three adopted cosmology, together with their 68.3% 
confidence interval. It also shows how the HDF correlation length changes if one uses the top-hat 
and synthetic redshift distribution functions discussed above, instead of the fiducial one. It can be 
seen that the uncertainty on N(z) introduces a negligible effect (given the present random errors) 
on the measure of ro, and regardless of which N(z) is adopted, the correlation length of the HDF 
sample is always « 3 times smaller than that of the brighter PHOT sample. 



5. RESULTS 

Figure 5 plots the correlation length of the three samples as a function of both the limiting 
magnitude and the corresponding absolute magnitude (in the A cosmology), and shows the main 
result of this study, namely that the clustering strength of Lyman-break galaxies is a function 
of their rest-frame UV luminosity at A ~ 1700 A. Remembering that this is a good tracer of the 
instantaneous rate of star formation (modulo obscuration by dust), another way to state the result 
is that LBGs with higher star-formation rates are more strongly clustered in space. 

Note that, strictly speaking, the SPEC and PHOT samples are not completely statistically 
independent, since the galaxies of the former also belong to the latter. However, in regard to the 
measure of the correlation length they may be considered as independent. In the case of the SPEC 
sample, the correlation length has been derived from the fluctuations of the counts in very large 
cubic cells, exploiting only the information on the redshifts of the galaxies with no consideration 
to the relative angular displacements between them. This information, on the other hand, has 
been used to measure the function w(6) of the PHOT sample, which also contains twice as many 
galaxies as the SPEC one. The redshifts have only been used in a cumulative way during the 
measure of the correlation length of the PHOT sample, as the distribution function N(z) in the 
Limber integral of Eqn.4.4. 

How significant is the detection of clustering segregation? Taken at face value, the data shown 
in Figure 5 suggest that we have likely detected it, despite the relatively large random errors. A 
X 2 test computed under the null hypothesis that ro does not depend on mu m (using the weighted 
average of the three measures as an estimate of the true value of ro) yields Xr = 4.44, 5.32, and 
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5.40 with 2 degrees of freedom. This implies that the null hypothesis is rejected at the 98.82%, 
99.51% and 99.55% confidence level in the three cosmologies, respectively. 

Systematic errors, however, can affect the measure of w(6). The PHOT and HDF samples 
have only partial redshift completeness and they might contain a fraction of interlopers at 
significantly lower redshifts than the rest of the LBGs. The inclusion of a fraction / of interlopers, 
which are spatially uncorrelated with the Lyman-break galaxies and hence essentially distributed 
at random respect to them, causes the observed estimate of w{9) to be underestimated by a factor 
(1 — f) 2 . However, this is unlikely to be a problem. As discussed in G98, the overall efficiency of 
the Lyman-break technique in going from photometrically selected candidates to spectroscopically 
confirmed LBGs is > 75%, with the redshifts of the remainder 25% undetermined. Hence, the 
interloper contamination of the PHOT sample (and presumably that of the HDF sample as well) 
is / < 25%. However, the true contamination is likely to be significantly smaller, since interlopers 
were never found among the positive identifications, and all the missed identifications were due 
to inadequate S/N. These were probably caused by observational accidents, such as astrometric 
errors or small misalignments of the multi-object masks of the spectrograph with the selected 
targets that resulted in insufficient flux being recorded on the detector. 

A more serious problem is the possibility that the observed correlation lengths are affected by 
systematic error due to cosmic variance, if the samples are not fair representations of the large 
scale structure at z ~ 3. As mentioned earlier, this results in a bias on the measured amplitude of 
w(9) (and on the volume density) induced by structure fluctuations on scales that are comparable 
or larger than that of the sample (integral constraint). This is unlikely a major problem for 
the SPEC and PHOT samples, since they probe relatively large volumes of space and consist of 
several, independent sub-samples from different regions of the sky, but it might affect the much 
smaller HDF sample. For example, Steidel et al. (1999) noticed that removing the criterion 
^300 — -B450 > 1-6 from the definition of LBGs in the HDF changes the estimated effective volume 
by only ~ 14%, but increases the total size of the sample by 45%, significantly altering the shape 
of the luminosity function. The excess galaxies have UV colors consistent with the possibility 
that they all belong to a single structure at z ~ 2. This shows the potential dangers of working 
with samples covering small volume of space such as the HDF. Fortunately, the inclusion of the 
criterion yields a luminosity function that matches fairly well that of the ground-based sample 
(defined through the same photometric criteria), both in normalization and faint -end slope, and, 
as noticed by S99, a Schechter fit to the ground-based data alone provides a good description 
of the HDF data as well. Also, we found that the correlation function of the HDF sample is 
insensitive, within the error, to the exclusion of the color criterion. 

The difference of tq between the SPEC and PHOT samples is small enough (given the errors) 
that alone it does not provide any information about the clustering segregation (the la error 
bars almost overlap). The evidence comes from the measure of ro in the HDF sample, and in 
order to assess the robustness of the result, we have used numerical simulations to estimate the 
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confidence level that the HDF is indeed less correlated than the PHOT sample. f\ Specifically, 
we have generated mock HDF samples that are realizations of a parent population with assigned 
correlation function, and we have measured their w{9) following exactly the same procedure used 
for the real data, including averaging over two mock samples to simulate the HDF-N and HDF-S 
observations We have generated a series of large galaxy distributions by performing Poisson 
sampling of 2-D lognormal density fields that have power spectrum corresponding to the assigned 
correlation function, assigning a probability of finding a galaxy in a given point as proportional to 
the density field in that point. We have normalized the probability so that the number density of 
the parent population of the mock galaxies has an assigned value, which we taken equal to the 
value observed in the HDF (Eqn.2.5). To simulate the effects the density fluctuations on large 
scales, we have allowed the size of each galaxy distribution to be much larger than the size of the 
HDF (we used 60 x 60 square arcmin), and in each case we have randomly extracted from this 
distribution a sample with the same geometry of the HDF. Thus, by construction, the "measured" 
w(9) of each mock sample is an estimate of the parent population's w(9), and it is subject to the 
combined effects of random errors and bias by integral constraint as the real data. 

In the null hypothesis that there is no clustering segregation, the HDF and the PHOT samples 
have the same parent angular correlation function. To estimate how confidently the data allow 
us to reject this hypothesis, we have used the power-law fit to the observed w{9) of the PHOT 
sample as input to the simulations and compared the distribution of the measures of w{9) in the 
mock samples with the observations in the HDF. Note that this is a conservative approach, since 
the observed w{9) is an underestimate of the parent correlation function of the PHOT sample 
because of the integral constraint bias. The top panel of Figure 6 shows the distribution of the 
values of w{9) of the mock samples with 1 < 9 < 2 arcsec together with the observed HDF value. 
According to the simulations, the null hypothesis can be rejected at the 98.4% confidence level. 

Since the range of angular separations where the HDF and PHOT correlation functions are 
reliably measured do not overlap, however, the above simulations can only test that the HDF is 
not clustered as strongly as the small-scale extrapolation of the PHOT w(9). Therefore, we have 
conducted a second test, where we have measured the correlation function of the HDF sample 
and of the mock samples in very large bins, 25 arcsec in size, to provide partial overlapping of 
angular separations with the PHOT sample. The bottom panel of Figure 6 shows the distribution 
of the simulated values of w{9) at 9 = 25 arcsec together with the observed value, while Figure 
7 shows the observed correlation functions of the PHOT and HDF samples as well as the best 
fit power-law models. Note that the error bars of the HDF data points are smaller than those 
in Figure 4, because of the larger number of pairs in the 25 arcsec bins. In this test too, the 
simulations suggest that the null hypothesis can be rejected at the 98.6% confidence level. 

In conclusion, the simulations increase our confidence that we have detected clustering 



3 For the simulations, we have used a code written by Cristiano Porciani, who generously made it available to us, 
that we have adapted to our specific problem. 
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segregation in the LBG population at z ~ 3. While we regard the detection as tentative — larger 
samples are needed to improve upon the accuracy and precision of the current measures — the 
simulations suggest that the samples selected with the criteria of Eqn.2.4 are at least not grossly 
unrepresentative of the z ~ 3 LBG population at faint magnitudes, and the value of r§ that we 
have measured is not largely affected by systematics. 

6. DARK MATTER HALOS AND LYMAN BREAK GALAXIES 

In this section we will discuss the implications of the detection of clustering segregation and 
how we can use it to derive information on the relationship between the activity of star formation 
and the mass of the galaxies. 

The clustering strength and abundances of virialized structures that form via gravitational 
instability ("halos" hereafter) depends to a large extent on their mass, with more massive 
halos being more strongly clustered and less abundant in space than less massive ones (e.g. 
see Mo &; White 1996). The UV morphology of LBGs (Giavalisco et al. 1996a; Steidel et al. 
1996b; Lowenthal et al. 1997) suggests that some degree of virialization has taken place in 
these structures. The strong spatial clustering of bright LBGs suggests that these systems trace 
relatively massive halos, with comparatively few halos of small mass populating the bright samples 
(otherwise we would not observe the strong clustering), apparently implying a correlation between 
the UV luminosity and the mass. The detection of clustering segregation seems to provide 
additional empirical support to this idea. 

If the UV luminosity is somehow correlated with the mass, the clustering strength should 
obey the same scaling laws predicted for the halos. Thus, it is interesting to compare the clustering 
properties and abundances of the LBGs with the predictions of the gravitational instability 
theory. The problem is that the only visible halos are those hosting star-forming galaxies with 
UV luminosity and spectral energy distribution that satisfy the selection criteria and sensitivity 
of the observations, and to predict the clustering properties of the LBGs one needs to know the 
relationship between the mass and the UV luminosity. A tested model for this relationship is not 
available, since the relevant physics is poorly understood. Luckily, as we shall see, the scaling 
law of the clustering strength with the volume density is highly degenerate with the form of the 
mass-UV luminosity relationship. Thus, we will compare the observed clustering properties of 
LBGs to the theory using a simple phenomenological model fitted to reproduce the luminosity 
function and the scaling law of the correlation length with the UV luminosity. 

We have modeled the mass distribution with the CDM power spectrum and we have 
computed the mass spectrum of the halos using the Press-Schechter (1974, PS) formalism. We 
have implemented the formalism using the technique described by Mo & White (1996). We used 
a primordial power spectrum with spectral index n = — 1, and the transfer function given by 
Bardeen et al. (1986) with T = 0.25, the value measured from local galaxy surveys (Peacock 1997; 
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Dodelson & Gaztanaga 1999). We fixed the amplitude of the spectrum at z = by adopting 
the cluster normalization of Eke et al. (1996), with as = 1.0, 0.9 and 0.5 for our three adopted 
cosmologies. 

The volume density and clustering strength of halos of mass M are controlled by the fractional 
overdensity at the collapse, u = 5 c /a(M), where 5 C ~ 1.7 is the linear overdensity of a spherical 
perturbation at collapse (e.g. Peebles 1980) and a(M) is the rms density fluctuation of the density 
field smoothed using a spherical top-hat window enclosing the mass M. For a Gaussian density 
field, the volume density of the halos is approximately given by 

n(M)dM = /^e- 1/2 / 2 -^-«, (6.1) 
y J V 7r M dM y ' 

where p is the average mass density of the universe. 

The spatial clustering of the halos, to first order, is amplified respect to that of the mass 
by a factor ~ v 2 and independent of the spatial scale (Kaiser 1984; Mo & White 1996), and the 
correlation function of the halos is proportional to that of the mass through the second power of 
the linear bias parameter b: 

Sh(r) = b 2 x £ m (r), (6.2) 

where to first order 

b~l + (v 2 -l)/5 c . (6.3) 

In this linear bias model, the clustering of halos of a given mass is entirely specified by the value 
of b(M), because the £ m (r) is fixed once the power spectrum has been assigned. Note that there 
are no free parameters in the model other than those necessary to specify the cosmology and the 
power spectrum, namely eg, T, and the spectral index n. 

A comparison to N-body simulations shows that Eqn.6.3 is accurate for halos with mass 
M £ M*f\, but it progressively over-predicts the bias on smaller scale (Mo & White 1996). Jing 
(1999) proposed a revised formula, derived from a fit to the results of N-body simulations, that 
it is accurate to better than 15% in the range M ^ 0.01M* (see also Porciani, Catelan & Lacey 
1999). The mass range and distribution of the halos used for the comparison with the LBGs are 
set if one requires that the samples of halos have the same clustering strength and volume density 
of the galaxies; when carrying out such comparisons, we used Eqn.6.3 as well ass the analogous 
ones made with Jing's approximation, finding similar results. 

Another important point is that Eqn.6.2 is not valid over spatial scales smaller than the 
linear size of the halos, because of spatial exclusion effects. N-body simulations (e.g. see Figure 
6 of Mo & White 1996) show that Eqn.6.2 begins to over-predict the autocorrelation function 
of halos (with M ^ M#) in Eulerian space (the observed one) for separations of the order of the 



4 M» is the mass scale of non-linearity, denned as ct 2 (M„) = 1. At z = 3, A/* = 4.4 x 10 12 , 2.8 x 10 11 , 6.5 x 10 s , 
respectively, in the three cosmologies adopted here. 
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Eulerian radius. We can estimate this from the relation M = 4/3tt5 c ()R e , where for a spherical 
perturbation S c is known (Bryan and Norman 1998) and p is the background density, and have an 
idea of the spatial (and angular) scales over which exclusion is important. The mass spectrum of 
LBGs is currently unconstrained, however the dependence of Re with the mass is very shallow 
and we can get a relative good answer using the few available estimates of the mass of bright 
galaxies by Pettini et al. (1998). These are in the range 10 10 to 10 11 M Q , and are likely lower 
limits to the true values. As we shall see later, spatial abundances and clustering properties of our 
three samples also suggest masses in the range 10 11 to 10 12 M for the SPEC and PHOT samples 
and 10 10 to 10 11 M for the HDF. If M = 10 10 (10 12 ) M , R E ~ 36 (170) fr _1 kpc in comoving 
coordinates, corresponding to a projected angular separation on the sky 9 ~ 2.5 (12) arcsec. 
Hence, exclusion effects do not seem to be a factor in the measure of w(9) of the ground-based 
samples (see G98). They also seems unlikely in the HDF sample, since of the 163 pairs with 
9 < 10 arcsec (where the bulk of the signal is observed, see Figure 4), only 15 have 9 2.5 arcsec. 
Of these, only a fraction of the order of [i?^/L e //) x (1 + £(Re)] ~ 15% (£ e // ~ 150 /i _1 Mpc is 
the effective linear size of the survey along the line of sight) is actually expected to have physical 
separations r ^ Re, namely only about 2 pairs. 

It is convenient to compare the clustering and halos and galaxies using the bias instead of the 
correlation length, because in this way, both the data and the models depend in a similar way by 
the choice of the normalization of the power spectrum and, to first order, the comparison is not 
affected by the uncertainties on ag. For the same reason, however, the comparison is also largely 
degenerate respect to the choice of cosmological parameters, namely to the absolute amplitude of 
the power spectrum at a given epoch. 

Equations 6.3 describes the "monochromatic" bias of halos with mass M. The average bias 
of a mass-limited sample of halos with mass M > Ml and mass function n(M) is given by 

ff? dMn(M)b(M) 
(b) = ^ — — — — , (6.4) 

nd 

where = dM n(M) is the mean volume density of the sample, and the expectation value 
of the correlation function of the halos of the sample is, to first order, related to the mass 
autocorrelation function by the usual relation ^(r) = (b) 2 ^(r) (Porciani et al. 1998). In the 
following we will refer to the function (b) = (b(nd)) as the clustering segregation function. 

In practice, the average bias of the LBGs of the SPEC sample is estimated from the ratio of 
the observed variance of galaxy counts to the variance of the mass fluctuations on the same scale 
(see A98): 

( b ) = \HtT5 — v ( 6 - 5 ) 

V a m\ K cell) 

while for the PHOT and HDF samples it is given by the ratio of their correlation function to that 
of the mass: 
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It is important to realize that these expressions are relative to different spatial scales, namely 
several Mpc for the SPEC and PHOT samples and ~ 1 ft^Mpc for the HDF one. Observations in 
the local universe (Peacock 1997) suggest that galaxy bias varies relatively slowly with the scale. 
However, while limiting our analysis to the scale-independent linear bias is probably appropriate 
with the present accuracy, one must keep in mind that the scale dependence of the bias at z ~ 3 is 
currently unconstrained (see also. Mann, Peacock & Heavens 1998; Somerville et al. 2000; Chen 
& Ostriker 2000). 

Finally, since Eqn.6.5 and 6.6 probe different spatial scales, they are also sensitive in different 
ways to the presence of non-linear clustering. Eqn.6.5 provides a measure of the bias averaged 
on scales that at z ~ 3 are only mildly non-linear or not linear at all (depending on the adopted 
cosmology), while Eqn.6.6 probes scales where relatively strong non-linear clustering can have 
already developed at z ~ 3, particularly in an open cosmology. Such non-linearities result in an 
overestimate of the linear bias above, if only the linear part of £ m (r) is used. This systematics 
would affect the open cosmology more than the flat ones, because for these £ m (r) enters the 
non-linear phase sooner in redshift than in the other two cases. Therefore, we have computed 
€m(r) taking the Fourier transform of the non-linear power spectrum, following the prescription 
of Peacock (1997). 

The bottom panel of Figure 8 shows the clustering segregation function of the LBGs (data 
points) compared to that of the CDM halos (continuous curve) for the A cosmology. The plot 
shows that, within the errors, the LBGs have the same clustering segregation function of the halos 
(similar results are valid for the other two cosmologies). To understand what such an agreement 
means and to actually compare the observations to the predicted segregation function of the 
galaxies (as opposed to that of the halos) we need to model the relationship between mass and 
UV luminosity, which is the subject of the next section. 

6.1. The Mass— UV Luminosity Relationship 

The luminosity at ~ 1700 A of LBGs is mostly the result of their star formation rate and the 
amount of intrinsic dust obscuration that the radiation suffers in situ, before leaving the galaxies. 
Physical arguments suggest that the mass is an important parameter in determining the star 
formation rate of the galaxies (e.g. Frenk & White 1995; A98). However, it is also reasonable to 
expect that both dust obscuration and star formation activity are characterized by some amount 
of stochasticity, even in galaxies with the same mass. For example, a different angular momentum 
(e.g. Dalcanton, Spergel & Summers 1997; Heavens & Jimenez 1999), density profile (Kennicutt 
1998a, b), or interactions with other nearby systems could result in LBGs with equal mass having 
different star formation rate. Or galaxies with the same mass and star formation rate could have 
different dust obscuration and, thus, different UV luminosity. Therefore, we have assumed that 
the UV luminosity Ljjy of LBGs hosted in halos of mass M is a random variable that depends 
on M through its mean {Luv) = £(M) and variance cr^y(M), and we have modeled Ljjy as the 
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product of C(M) with a random variable A 

L UV = AC(M), (6.7) 

where the mean value of A is, by definition, equal to 1. For simplicity, we have assumed in the 
following that the distribution function of log (.A) is Gaussian. 

Before fitting the model mass-UV luminosity relationship to the data to simultaneously 
reproducing the clustering strength, luminosity distribution and spatial abundances of LBGs we 
have to select the observables. Luminosity and abundances are constrained by the luminosity 
function (f>(Ljjy). As to the clustering properties, the clustering segregation is not very useful 
because, as we will see in a moment, it is largely degenerate with the shape of £(M). Much 
more useful is the function (b) = (6(Lo)) that links the average bias of a sample of LBGs to its 
limiting luminosity Lq. This depends on both afj V and £(M), since these together set the mass 
range and distribution of the galaxies that enter the sample. Computing the predicted observables 
is straightforward. The probability to find a galaxy hosted in a halo with mass M that has 
luminosity L = Ljjy{A,M) is 

tt(A, M) dA dM = n(M) N g (M) f(A) 5{L - L UV (A, M)) dA dM (6.8) 

where N g (M) is the average number of galaxies hosted in halos of mass M and f(A) is the 
distribution function of A. The average bias of a flux limited sample of LBGs with L > Lq is given 
by: 

{be{Lo)) = jr P (M\L ) d M ' (6 - 9) 

where 

r+oo r+oo r+oo /dLnv\ L 

p(M\L )=j L dLj_^7r(A,M)dA = n(M)j L f(A)\-^j dL (6.10) 

is the probability that a halo with mass M is included in the sample, and the partial derivative is 
computed for Ljjy{A,M) = L. The luminosity function is given by: 

roo roo roo / ftj rrT , \ ~ 1 

HL)=J o dM J ir(A,M)dA = J^ f^p) f (A) n(M) N g (M) dM . (6.11) 

It is also interesting to compute the average mass and the variance of a sample of galaxies with 
luminosity L, which are given by: 

, / ^ Jn°° Mi^^y 1 f(A)n(M)NJM)dM 

<M(L)) = ^ V 9A ' — J ) ' K ' gK ' , (6.12a) 

<p[L) 

and 

2 ( n v _ Io°°( M ~ ( M ( L ))) 2 (^r 1 M n(M) N g (M) dM 
a M\ L )l — ^Tyj • (b.i-Zb) 



In the following we have assumed N g (M) = 1; as we will discuss later, it is unlikely that this 
approximation has grossly misdirected our results. We have experimented with 3 different models 
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to study how the functional form of C(M) and afjy(M) affect the observables. Model A has 
C(M) = eM a and afjy = <7q, and hence three free parameters, namely a, e and a. In model B the 
variance is a function of the mass, and for simplicity, we worked in the "logarithmic space" f], and 
we modeled the variance of the variable log^) as a 2 = a 2 [1/1 + (M/M a )f. Model B has four free 
parameters, namely a, e, ao and M a . In model C the variance is again constant, but the slope of 
the power law varies from the high-mass asymptotic value a to the low-mass one /3, and is equal 
to (a + pi)/ 2 at the cut-off mass M co . We used the arctg law to model this variation. This model 
has five free parameters, namely a, P, M co , e, and cjq. 

We have fitted the models to the data using the multi-dimensional algorithm "AMOEBA" 
(Press et. al. 1994) to minimize the sum x 2 = Xc + xi °f t ne t wo chi-square's relative to the 
function (6(Lq)} an d <f>(L), respectively. The data on the luminosity function come from the LBGs 
of the PHOT and HDF samples, after dimming the HDF magnitudes by the amount A1Z = 0.25 
(S99) to account for their smaller mean redshift, according to the prescription by S99. When 
computing the x\i we did not include the two faintest data points, because no corrections for 
incompleteness have been computed for the HDF sample, and these are very likely the ones more 
strongly affected. 

To a good extent, the overall slope of <j>(L) is controlled by the logarithmic slope of C(M), 
while the curvature depends on ofjy. If afjy is constant, increasing its value reduces the curvature. 
If afjy increases towards smaller masses, the curvature of 4>{L) increases (interestingly, a afjy that 
increases with M is inconsistent with the data, since it results in a luminosity function with the 
wrong curvature). Increasing the logarithmic slope of C{M) makes the function (6(Lo)) steeper, 
while increasing the variance flattens it. The latter effect is very pronounced at the high-end of 
the mass spectrum (and of the luminosity function), but it becomes progressively less noticeable 
at smaller masses (and fainter luminosities). 

The top two panels of Figure 8 show the results of the fits in the case of the A cosmology, 
while the bottom panel shows the clustering segregation function of the galaxies computed from 
the fitted model, compared to the analogous quantity for the halos (continuous curve) and to the 
observations (data points). The dotted curves represent model A, the short-dashed curves model 
B, and the long-dashed curves model C. Similar plots are obtained in the other two cosmologies. 
However, while the quality of the fits in the case of the open cosmology is similar to the case 
discussed above, the fits are somewhat worse in the Einstein-de Sitter world. We plan to return 
to the comparisons of the results of the fits in the various world models in a forthcoming paper. 

Model A yields a slope a ~ 1.5 and a small variance {a ;$ 0.2), and as the figure shows, it 
fits the observed (6(Lo)) data points relatively well, but only crudely reproduces the shape of the 
luminosity function. Smaller values of a would give a better agreement at the high-luminosity 
end, but a larger discrepancy at the low-luminosity one, while a larger slope would have the 
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opposite effect. The clustering segregation function predicted by this model is very similar to 
that of the halos (because of the very small scatter). Increasing Co decreases the curvature of 
the luminosity function and it decreases (6(Lq)), reducing the quality of the fit. Thus, while a 
power-law (with the CDM + PS mass spectrum) is consistent with the clustering properties of 
LBGs, it is inconsistent with the luminosity function. 

By allowing the scatter to increase at the low-mass end of the spectrum, model B improves 
the fit to the luminosity function without significantly changing the agreement with the data on 
(b(L )). We found a ~ 1.0 and a ~ 7.5. This corresponds to a ~ 0.7 for M = 10 12 M , and 
a ~ 3.8 for M = 10 11 M Q . What happens is that LBGs hosted in halos of lower mass become 
increasingly less likely to be bright enough to be included in the sample, progressively decreasing 
4>(L) at the faint end, and thus increasing its curvature. This effect combines with the shallow 
dependence of the bias on the mass for small masses to make the clustering segregation function 
of the LBGs predicted by this model highly degenerate respect to the parameters of the mass-UV 
relationship, and essentially identical to that of the halos. Note that this model predicts that the 
clustering strength reaches a maximum at the bright end and then it decreases again. This is 
because the bias is a steep function of the mass for large masses, and the fitted variance is large 
enough to decrease the average bias of very bright samples. 

The variable slope of the power-law in model C allows for a better fit to the luminosity 
function than model A. We found a high-mass slope a ~ 0.8 and a low-mass slope (3 ~ 2, with 
a ~ 0.7. Note that accurate measures of the correlation length of bright samples will be very 
effective to discriminate among the various models. 

We note that error bars on the fitted parameters, estimated by bootstrapping the data points 
with gaussian distributions, are in general relatively small, with the 68% confidence level fractional 
errors in the range 30-50%. This is due to the fact that the fits are dominated by the data points 
of the luminosity function. There are cases, however, when the errors can be very large, such as 
the error on do in model A, which is ~ 1000%. The reason is that in this model the function 
(b(Lo)) has a stronger dependence on oq than 4>{L) has, and thus the error is dominated by the 
relatively large uncertainties on the clustering data. In this case the fit only yielded an upper limit 
to the value of the scatter (do rS 0.2 from 1,000 bootstrapped samples). 

Finally, although we do not discuss in detail the results of the fits, it is important to keep 
in mind that the mass-UV luminosity relationship that we have derived here includes the effects 
of dust obscuration, and thus it will be different, in general, from the relationship between the 
mass and the intrinsic UV luminosity (or, equivalently, the star formation rate). The amplitude 
of the power law (or power laws) used to model the observed average luminosity is smaller, while 
the variance is larger, because it includes the contribution of the fluctuations of extinction from 
galaxy to galaxy. The slope of the power law will be also be different if the obscuration properties 
of the galaxies depend on their luminosity. 
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7. DISCUSSION 

The primary results of this work are the following: 1) the clustering strength of LBGs 
scales with their UV luminosity, with brighter galaxies being more strongly clustered in space; 
2) the corresponding scaling law of the clustering strength with the volume density (clustering 
segregation) is, within the observational uncertainties, the same as that predicted by the 
gravitational instability for CDM halos, regardless of the values of the cosmological parameters 
that we have considered; 3) the predicted clustering segregation of LBGs depends very weakly 
on the relationship between the UV luminosity of the galaxies and their mass. Models of the 
mass-UV luminosity relationship that simultaneously reproduce the luminosity function of the 
LBGs and the scaling law of the clustering strength with the UV luminosity predict a clustering 
segregation function that is very similar to that of the halos and to that of the observed LBGs. 

In the models we have assumed N g (M) = 1, namely that the average number of LBGs per 
halo is unitary and independent on the mass of the halo. While this is almost certainly not true in 
detail, this approximation seems adequate here. We actually only need to discuss the possibility 
that N g (M) > 1 in some region of the mass spectrum, since the models already include the case of 
"invisible" halos, i.e. halos whose LBGs are too faint to be observed, but they do not include the 
case of halos with multiple LBGs. Physically, one expects that N g (M) is a monotonic increasing 
function of the mass, reflecting the fact that massive halos are more likely than smaller halos 
to have substructure and be host to more than one galaxy that satisfies the selection criteria 
of the observations. The question is whether or not the portion of the mass spectrum covered 
by the observations includes halos for which "multiplicity effects" have observable consequences. 
It could be, for example, that such halos are relatively rare at z ~ 3 and the few (if any) that 
might be present in the survey do not significantly contribute to the observed correlation function. 
Empirically, there are no obvious indications of multiplicity in the samples, and the approximation 
that a suitable halo is populated by at most one bright LBG is consistent with the observed spatial 
and angular distribution of the galaxies. Among pairs of galaxies in the SPEC sample with the 
same redshift only two have angular separation smaller than 20 arcsec and only one at 10 arcsec 
(at z = 3, 10 arcsec correspond to 224, 216, 144 h~ l kpc in comoving coordinates, respectively). 
In the PHOT sample, the number of close pairs (9 < 20 arcsec) is consistent with the Poisson 
statistics (see G98) and, if anything, it actually seems to be smaller than the extrapolation of 
the observed angular correlation function, which is measured over scales 9 ^ 20 arcsec (G98). A 
similar situation exists for the HDF sample, as we discussed in Section 6. Of course, since the 
HDF sample does not cover enough area to include a sufficient number of bright LBGs (only 45 
galaxies in the HDF sample have 1Z < 25.5), the present data do not constrain the possibility that 
galaxies fainter than the PHOT flux limit can reside in close spatial proximity of brighter ones. 

The conclusion that seems to emerge is that, within the accuracy of the current observations, 
the clustering properties and abundance in space of LBGs are consistent with those of halos that 
have formed by the epoch of the observations. These galaxies apparently flag the sites of the 
halos with high efficiency, and in a way such that the bias of the UV light relative to the mass is 



-27- 



similar to that of the halos relative to the mass. This provides empirical support to the paradigm 
that galaxies form first in the highly biased virialized peaks of the mass distribution, and that 
gravitational instability is the primary physical process behind the formation of cosmic structures. 
It is also interesting that these results are insensitive to the choice of the cosmological parameters 
(at least given the accuracy allowed by the present data). The clustering and abundance of 
LBGs seem to be more effective in constraining the mechanisms of galaxy formation than in 
discriminating among the background cosmologies. 

This study has also given us some insight on how the clustering properties and the luminosity 
function of the LBGs depend on the relationship between the mass and the UV luminosity 
(assuming the CDM halo mass spectrum). Three interesting points come out of the analysis. 
First, a mass-UV luminosity relationship similar to a power-law and a constant scatter (i.e. a 
variance independent on the mass) do not seem consistent with the data. This suggests that the 
relationship between the star formation activity and the mass is not scale invariant along the mass 
spectrum of the observed LBGs. Either the overall efficiency of star formation per unit mass, or 
the scatter between mass and star formation, or both, seem to increase towards the low end of the 
spectrum. Adelberger (2000) suggested that a steepening of the mass-UV luminosity relationship 
towards lower masses may result from the increased importance of feedback mechanisms in the star 
formation activity, such as supernovae and stellar winds. Similarly, a scatter increasing towards 
lower masses may be the signature of the growing importance of the environment, for example 
merging and interactions, over the local gravity on the star formation activity of small galaxies. 
The fits suggest that the value of M co in model C, namely the mass that marks the transition 
from the high-mass slope to the low-mass one in the mass-UV luminosity relationship, is ~ 10 12 
M in the open and A cosmology and ~ 10 11 in the EdS one, while the value of M a in model B 
is ~ 10 11 and 10 10 M in the same cases, respectively. 

The second interesting suggestion is that the scatter of the mass-UV luminosity relationship, 
namely the variance of the UV luminosity of galaxies of given mass, is relatively small at the 
high-mass end of the spectrum. This came out consistently, regardless of the adopted model. For 
example, the scatter of galaxies with luminosity (expressed in magnitude) 1Z = 25.5 (the flux limit 
of the PHOT sample) is ajjy ^ 20% in model A and ~ 70% of the average value in model B and 
C, respectively. On the contrary, the scatter at the small-mass end depends on the adopted model, 
and thus it remains unconstrained. For example, galaxies with 1Z ~ 27.2 have o\jy ~ 250% in 
model B, but auv ~ 70% in model B. To this purpose it is interesting to observe that the "Global 
Schmidt Law" for local starburst galaxies discussed by Kennicutt (1998b) shows that the scatter 
between the integrated star formation surface density and the integrated gas surface density 
remains relatively constant in absolute value over a range that spans ~ 6 orders of magnitudes in 
gas and star-formation densities (hence, the relative scatter increases for lower gas density). If 
the gas mass fraction traces the total mass, the Schmidt Law would favor a situation similar to 
model B. As we noted earlier and as Figure 8 (top panel) shows, more accurate measures of the 
correlation function at bright luminosity will be very valuable to discriminate among the models. 
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The possibility that the scatter for bright LBGs is relatively small is an intriguing one, since 
it implies that these galaxies flag the sites of massive halos with good efficiency, with the UV 
light being a tracer of the total mass and star formation a process regulated more by local gravity 
than by external factors such as merging and interactions. This also argues against a significant 
population of massive and "dark" halos being under-represented in UV-selected samples, either 
because without appreciable star formation or because heavily obscured by dust. Samples of bright 
UV-selected galaxies at high redshifts seem to trace the sites of massive halos with high efficiency, 
and there is no evidence that the most massive and actively star forming of these systems might 
be systematically missing. Note that Adelberger & Steidel (2000) reach similar conclusions from 
independent arguments based on the sub-millimetric background and faint counts. 

Another interesting consequence is that one can constrain the duty cycle of the UV- 
bright phase. If most halos are being simultaneously observed, the duty cycle must be, on 
average, comparable or larger than the cosmic time covered by the observations, namely 
Tuv ~ ATlbg ~ 0.5, 0.6, 0.3 h" 1 Gyr. A significantly shorter duty cycle, for example ~ 10 7 yr, 
would imply an intrinsic volume density of halos ~ 50 times higher than that of LBGs. In turn, 
the duty cycle gives information on the stellar mass associated with each halo. For example, the 
star formation rate of LBGs at z = 3 with 1Z = 23 is ~ 50, 30, 20 /i~ 2 Mq yr -1 , uncorrected for 
dust obscuration. Thus, the average stellar mass associated to such halos is ^ 2.5, 2.0, 0.6 
10 10 h~ 3 M Q . With dust corrections likely to be a factor of several (Calzetti 1997a; Dickinson 
1998; Pettini et al. 1998; S98; Steidel & Adelberger 2000), these lower limits become close to 10 11 
/i~ 3 M , comparable to the stellar mass of present-day galaxies of ~ L* luminosity. 

This does not mean that the average duty cycle of the individual LBGs is £ 0.5 Gyr, but only 
that the hosting halos are made "visible" for such a period of time. For example, semi-analytic 
models (Kolatt et al. 1999; Somerville et al. 2000) predict a series of LBGs with much shorter duty 
cycles associated to the same halo which become UV-luminous in sequence, on average one at a 
time, as a result of interactions. It would be interesting to know what these models predict for the 
clustering segregation and scatter of the mass-UV luminosity relationship. Direct constraints on 
the duty cycle of the individual galaxies based on their spectroscopic and photometric properties 
are still very uncertain (e.g. Lowenthal et al. 1997; Sawicki & Yee 1997; Pettini et al. 1999; 
Dickinson 2000). Regardless of the duration of the star formation of the individual galaxies, 
however, the clustering properties and the abundances of LBGs seem to imply that a substantial 
amount of stellar mass has been produced at z ~ 3 in close spatial proximity to the brightest of 
them. Note that both the estimate of the stellar mass and the mass of the halos (as we shall see 
later) change by an order of magnitude with the cosmologies adopted here. Thus, the implied 
mass-to-light ratios (where the light is now the one at visible wavelengths emitted by the formed 
stellar population) predicted for these systems at the present time remain approximately constant 
to ~ 10 times solar, consistent to what observed in present-day bright galaxies (see Fukugita, 
Hogan k Peebles 1998). 

Finally, the third interesting point is that having fitted the mass-UV luminosity relationship 
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to simultaneously reproduce the dependence of the clustering strength with the luminosity and 
the luminosity function, one can assign a scale of mass to the LBGs for a given choice of the 
cosmological parameters. It is useful (from the "observational" point of view) to express this in 
terms of the average mass and the variance for galaxies with assigned luminosity L (Eqn.6.12). 
Interestingly, while the variance depends on the adopted mass-UV luminosity model, and thus it is 
not constrained, the average varies weakly with the model. For example, the average mass and the 
standard deviation of galaxies with luminosity 1Z = 23, 25.5 and 27.0 according to model C in the 
A cosmology are M = 2.5 ± 2.7, 0.9 ± 0.7 and 0.4 ±0.12 x 10 12 M , respectively. As a comparison, 
the average from model A and B are 3.7, 0.8, 0.3 and 2.9, 0.8, 0.4 xlO 12 M , respectively, but 
the standard deviations are ~ 100 times smaller and ~ 50% larger, respectively. In the open and 
Einstein-de Sitter cosmologies the values of the average from model C are 6.2, 1.5 and 0.5 xlO 12 
M Q and 1, 0.5 and 0.3 xlO 11 M , respectively. 

Irrespective of the exact value, note that the variance of the mass (for given luminosity) 
decreases with decreasing luminosity, while we discussed earlier that the variance of the luminosity 
(for given mass) seems to decrease with increasing mass. The two effects are not in contradiction, 
and are due to the shape of the halo mass spectrum, which becomes increasingly flatter towards 
the low-mass end. The variance of the luminosity for a fixed value of the mass includes the 
contribution of only one type of halos with given volume density. The variance of the mass for a 
fixed luminosity, on the contrary, includes the contribution of halos from the whole spectrum. Since 
the average luminosity decreases with decreasing mass, the mass variance at fainter luminosity 
becomes increasingly more dominated by low-mass halos, which are much more abundant in 
space than the high-mass ones and for which the mass spectrum is flatter, and hence increasingly 
smaller. 

We conclude by pointing out that the size of the present samples is clearly the limiting factor 
in this study. We are currently conducting a wide-area survey for LBGs at z ~ 3, and we plan to 
return soon on the issues that we have discussed here with new measures of the correlation length 
obtained from much larger samples extracted from contiguous fields (0.3 square degree each). 

In summary: 

1) We have studied the dependence of the spatial clustering strength of Lyman-break 
galaxies at z = 3 with their UV luminosity at A = 1700 A. We used three samples of LBGs with 
progressively fainter limiting magnitude, two from our U n GTZ ground-based surveys and another 
one from the Hubble Deep Field (both North and South). 

2) We have found evidence that the correlation length decreases by a factor w 3 over the 
range of magnitudes that we have probed, namely 25 TZ ^ 27. Lyman-break galaxies with 
higher 1700 A luminosity seem to be more strongly clustered in space, suggesting that fainter 
samples include galaxies with smaller mass. 

3) The observed scaling law of the correlation length is consistent with the predictions of 
the theory of gravitational instability and implies that fainter samples include less massive LBGs. 



-30- 



The scaling law of the clustering strength with the volume density is, within the errors, the same 
as the one predicted for mass-limited samples of halos. This is interesting, because this function 
is essentially independent from the relationship between the mass of the galaxies and their UV 
luminosity. The LBGs seem to flag the sites of halos with high efficiency and in a way such that 
the bias of the light respect to the mass is similar to that of the halos respect to the mass. We 
interpret this result as strong support to the theory of biased galaxy formation. 

4) Fitting models of the mass-UV luminosity relationship to simultaneously reproduce the 
observed clustering segregation and the luminosity function, suggests that mass and UV luminosity 
are relatively tightly correlated in bright LBGs. Galaxies with average luminosity 1Z = 25.5 have 
standard deviation auv ~ 70% of the average (the scatter is not constrained at fainter luminosity) . 
This suggests that i) the mass is an important parameter in regulating star formation in massive 
galaxies; ii) the duty-cycle of the UV-bright phase is similar to the cosmic time spanned by the 
observations (~ 0.5 Gyr), which in turn would set limits to the stellar mass assembled in the 
galaxies at the epoch of the observations; iii) the fraction of massive halos at z ~ 3 that have not 
started substantial star formation or whose UV light is not observed because heavily obscured 
must be relatively rare. 

5) The fits also show that a scale invariant relationship (e.g. a power law) between mass 
and UV luminosity is not consistent with the observations, suggesting that the properties of star 
formation of galaxies vary along the mass spectrum. From the fits one can also assign a scale 
of mass to the galaxies. For example, in the A cosmology the average mass of galaxies with 
luminosity TZ = 23, 25.5 and 27.0 A-cosmology (M) = 2.5, 0.9, and 0.4 x 10 12 M , respectively. 
These numbers would be « 2 times larger in the open universe and ps 10 times smaller in the EdS 
one. 
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Table 1. The Three Samples 



# 


Sample 






N c 


5 d 


z c 


1 


SPEC 


725.9 


TZ < 25.0 


446 


0.7 ±0.1 


3.04 


2 


PHOT 


718.9 


TZ < 25.5 


876 


1.22 ±0.18 


3.04 


3 


HDF 


9.2 


V 60 6 < 27.0 


271 


29.4 ±3.5 


2.60 f 



a Surveyed area, in units of arcmin 2 . 
b Limiting magnitude in the AB scale. 
c Number of selected Lyman-break galaxies. 
d Surface density, in units of arcmin -2 . 
°Mean redshift. 

From the simulated redshift distribution, see §4.1. 



Table 2. The Volume Density 8, 



Sample 


N(z) h 


n = 0.2, n A = o 


n = 0.3, n A = 0.7 


n = i.o, n A = o 


SPEC 


Observed 


1.9 ±0.3, 


1.8 ±0.3, 


7.2 ± 1.1 


PHOT 


Observed 


3.5 ± 0.5, 


3.3 ±0.5, 


13 ±2 


HDF 


Simulated 


46 ± 5, 


43 ± 5, 


169 ± 20 


HDF 


Top-Hat 


27 ± 3, 


24 ±3, 


94 ± 11 


HDF 


Synthetic 


36 ± 4, 


32 ±4, 


122 ± 15 



a Volumc density, in units of comoving 10- 3 /i 3 Mpc- 3 . 
b Redshift distribution function N(z). See §4.2 for details. 
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Table 3. The Parameters of the Correlation Functions 



Sample 




7 




r a 










n = 0.2, n A = o 


n = 0.3, n A = 0.7 


n = i, n A = o 


SPEC 


2. 


+0.2 
u -0.2 


4.2 ±0.6 


5.0 ±0.7 


3.1 ±0.4 


PHOT 


2 


+0.2 


o 1+0.7 


a 9+0.7 


I O+0-4 
1 ' y -0.5 


HDF 


2 


9 +0.6 
^-0.3 


1 1+0.9 


1 9+0.9 


n s+o.6 


HDF (TH) b 


2. 


9+0.3 
-0.2 


1 0+ ' 8 
1 - u -0.7 


i o+ - 8 


n 7 +0.5 


HDF (SYN) b 


2. 


9+0.3 
-0.2 


Q+ - 8 

u.y_ 7 


1 0+ ' 8 


6+ ' 5 



a Comoving coordinates, in units of h 1 Mpc. 

b TH and SYN denote the "top-hat" and "synthetic" N(z) in the Limber transform 
(see §4.2 and 5.2). 
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Fig. 1. — The redshift distributions of the ground-based and HDF Lyman-break galaxies. The 
thick continuous histogram is that of the TOTALSPEC ground-based sample with 1Z < 25.5. The 
thin-dotted and thin-dashed histograms show a bright sub-sample with 1Z < 24.5 and a fainter sub- 
sample with 24.5 < 1Z < 25.5, respectively. A Kolmogorov-Smirnov test shows that the two redshift 
distributions are consistent with being extracted from the same parent population, implying that 
over the range of magnitude considered here the function N(z) does not depend on luminosity. The 
shaded histogram shows the distribution of the 27 spectroscopic redshifts of the HDF sample. The 
thin continuous line shows the predicted N(z) for the HDF Lyman-break galaxies (in arbitrary 
units) obtained combining the intrinsic distribution of UV colors of the ground-based sample with 
the filters and selection criteria of the HDF survey (see §4.2). 
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Fig. 2. — Histogram showing the distribution of the redshifts in the Westphal field. The bin size 
is Az = 0.15. The continuous curve is obtained by smoothing the histogram of the redshifts of the 
TOTALSPEC sample (see text), and it represents the redshift selection function of the survey. 
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Fig. 3. — Number counts of Lyman-break galaxies in the 3 samples discussed in the paper. Only 
the PHOT and HDF samples are, strictly speaking, flux-limited and color-selected samples (see 
§2). However, the counts of the SPEC sample show that, for the purposes of estimating the 
volume density of the galaxies, this can also be treated in the same way as the other two. The 
magnitudes of the SPEC and PHOT sample are in the TZ band, those of the HDF sample in the Vqqq 
band. It is possible to estimate the TZ magnitudes of the HDF galaxies using the approximation 
ft = (^606 + ^su) /2, and the corresponding data points are plotted in the figure with a dotted line. 
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Fig. 4. — Top. Weighted average angular correlation function of the PHOT sample (from G98). 
The filled and open symbols represent the PB and LS estimators, respectively. For clarity, the error 
bars are shown separated from the data points. The continuous curves are the best fit power law 
to the data (PB: continuous line, LS dashed line). Bottom. Weighted average angular correlation 
function of the HDF sample. Although represented with separate symbols (as above), the PB and 
LS estimators are degenerate in this case, and cannot be distinguished in the figure. Again, for 
clarity, the error bars are shown separated from the data points. The continuous curves show the 
best fit power law to the data. 
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Fig. 5. — The correlation length of the three samples plotted against their apparent and absolute 
magnitudes. The letters "O" , "A" and "F" denote the open, A and Einstein-de Sitter cosmological 
models adopted throughout the paper. The absolute magnitudes have been computed in the A 
cosmology. For clarity, we separated the data points relative to different cosmologies by adding a 
constant value, as specified in the figure. 
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Fig. 6. — The Monte-Carlo simulations used to estimate the confidence level of the detection of 
clustering segregation. The histograms show the distribution of w(0) of 1,100 realizations of mock 
samples that have the same geometry and surface density as the HDF, but that are extracted from 
a parent distribution with the same correlation function of the PHOT sample (see text). As the real 
HDF sample, each measure of w(9) is actually the weighted average of two independent measures. 
The vertical dashed lines show the observed w(9) of the real HDF. Top. The realizations of w(9) of 
the mock samples in the interval 1 < 9 < 2 arcsec. The observed correlation function of the PHOT 
and HDF samples do not overlap at these angular separations, and the correlation function used 
in the mock catalogs is the extrapolation to small scales of the best fit power-law to the PHOT 
data. Bottom. As above, but with the angular separation in the range < 9 < 25 arcsec. By 
using such a large bin we can directly compare w(9) of the PHOT and HDF samples at the same 
angular separations. The simulations suggest that the null hypothesis that the HDF sample has 
the same correlation function as the PHOT one can be rejected at the ~ 98.5% confidence level. 
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Fig. 7. — A comparison of the observed angular correlation function of the PHOT sample (open 
points) and HDF sample (filled points). The data points are obtained in angular bins of 25 arcsec 
in size. Also plotted are the best-fit power law curves from the data shown in Figure 4. 
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Fig. 8. — The fit of the model of the mass-UV luminosity relationship to the data and the 
corresponding clustering segregation function, in the case of the A cosmology. Top: the mean 
bias {b) versus the limiting UV luminosity Lq. Middle: the luminosity function. Bottom: the 
clustering segregation function. The data points represent the observations of LBGs, while the 
broken curves shows the predictions based on the CDM power spectrum and the Press-Schechter 
mass distribution of halos after fitting the mass-UV luminosity models described in §7 to the 
(6(Lo)) function and the luminosity function. The dotted curves represent model A, the short- 
dashed curves model B, and the long-dashed curves model C. The continuous curve in the bottom 
panel is the clustering segregation function of the halos. The clustering segregation function of the 
LBGs is computed using the result of the fits. Very similar plots are obtained in the other two 
cosmologies. For the value of n p , the volume density of the PHOT sample, refer to Table 2. 



